Exercise: Sorting 1. ˆ Let A be any array. ˆ Write a program which outputs the sorted array of A (in ascending order).

Transcription

1 Exercise: Sorting 1 ˆ Let A be any array. ˆ Write a program which outputs the sorted array of A (in ascending order). ˆ For example, A = [5, 4, 1, 2, 3]. ˆ Then the sorted array is [1, 2, 3, 4, 5]. ˆ You may use sort. 1 See Zheng-Liang Lu 134 / 179

2 Practical Exercise 1 >> stocks = {'Google', 15;... 2 'Facebook', 12;... 3 'Apple', 18}; 4 >> [~, index] = sort([stocks{:, 2}], 'descend') 5 6 index = >> stocks = stocks(index, :) stocks = 'Apple' [18] 15 'Google' [15] 16 'Facebook' [12] Zheng-Liang Lu 135 / 179

3 Another Exercise: Shuffling ˆ Write a program which randomly permutes the elements of the input array A. ˆ For example, A = [1, 2, 3, 4, 5]. ˆ Then a possible permutation of A is [4, 2, 1, 3, 5]. ˆ Make sure your program correct in statistical sense. 2 ˆ Verify by Monte Carlo simulation. ˆ Write another program which enumerates all possible permutations of A. 3 2 Read 3 See salgorithm. Zheng-Liang Lu 136 / 179

4 1 >> A = 'abcdef'; 2 >> A = A(randperm(length(A))) 3 4 A = 5 6 dafbce ˆ You may use randperm(a) to generate an index array to permute the elements of A randomly. Zheng-Liang Lu 137 / 179

5 Analysis of Algorithms ˆ Given a problem, suppose that there exist various algorithms. ˆ Then we investigate these algorithms in various dimensions and choose the most appropriate one. ˆ Normally we want efficient algorithms. ˆ We now define the growth rate of the running time as a function of input size n, denoted by f (n). ˆ For simplicity, assume that every instruction takes a unit time. ˆ Can you find f (n) for each program? Zheng-Liang Lu 138 / 179

6 O-notation 4 ˆ In math, O-notation describes the limiting behavior of a function, usually in terms of simple functions. ˆ We say that f (n) O(g(n)) as n if and only if there is a constant c > 0 and a real number n 0 such that f (n) c g(n) n n 0. (1) ˆ So O(g(n)) is a collection featured by some simple function g(n). ˆ f (n) O(g(n)) means that f (n) is one instance of O(g(n)). 4 See Zheng-Liang Lu 139 / 179

7 ˆ For example, 8n 2 3n + 4 O(n 2 ). ˆ Moreover, 8n 2 3n + 4 O(n 3 ) but 8n 2 3n + 4 / O(n). Zheng-Liang Lu 140 / 179

8 Fundamental Functions for Growth Rate 5 5 See Table 4.1 and Figure 4.2 in Goodrich and etc, p Zheng-Liang Lu 141 / 179

9 ˆ We use O-notation to describe the asymptotic 6 upper bound of complexity of the algorithm. ˆ So O-notation is widely used to classify algorithms by how they respond to changes in its input size. 7 ˆ Time complexity ˆ Space complexity ˆ Note that we often make a trade-off between time and space. ˆ Unlike time, we can reuse memory. 6 The asymptotic sense is that the input size n grows toward infinity. 7 Actually, there are Θ, θ, o, Ω, and ω which are used to classify algorithms. Zheng-Liang Lu 142 / 179

10 Faster Is Better? ˆ Consider a 10-year deposit account. ˆ You deposit 10, 000 TWD in the beginning of each year. ˆ Assume that the bank pays an annual interest rate r = 10% compounded annually. ˆ Determine the balance at the end of 10 years. ˆ The answer is 175, TWD. Zheng-Liang Lu 143 / 179

11 Zheng-Liang Lu 144 / 179

12 Solution 1 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 6 s = 1; 7 for i = 1 : years s = s * (1 + r) + 1; 9 end 10 s = s * (1 + r) * 1e4 ˆ Time complexity: O(n) ˆ Space complexity: O(1) ˆ It is similar to Horner s method. Zheng-Liang Lu 145 / 179

13 Solution 2 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 6 s = 0; 7 for i = 1 : years 8 s = s + (1 + r) ˆ i; 9 end 10 s = s * 1e4 ˆ Time complexity: O(n) ˆ Space complexity: O(1) Zheng-Liang Lu 146 / 179

14 Solution 3 1 clear; clc; 2 3 r = 0.1; 4 years = 10; 5 s = (1 + r) * ((1 + r) ˆ years - 1) / r * 1e4 ˆ Time complexity: O(1) (Why?) ˆ Space complexity: O(1) ˆ Pros: fastest and compact; Cons: not robust Zheng-Liang Lu 147 / 179

15 Real Case: Floating Interest Rate ˆ Instead of constant interest rates, assume that the sequence of floating interest rates is r = 0.1 : 0.01 : ˆ Determine the compound amount. ˆ The balance is 125, TWD. Zheng-Liang Lu 148 / 179

16 Solution 1 clear; clc; 2 3 r = 0.1 : : 0.01; 4 years = 10; 5 6 s = 1; 7 for i = 1 : years s = s * (1 + r(i)) + 1; 9 end 10 s = s * (1 + r(end)) * 1e4 ˆ Trade-off: efficiency and generality Zheng-Liang Lu 149 / 179

17 Why Numerical Methods? ˆ Very few problems, however, have simple and analytical solutions. ˆ Numerical methods provide computational solutions for many crucial problems, thought the correctness of the numerical methods remains a big issue. 8 ˆ Simulation techniques 9 significantly reduce the cost both in industry and science. ˆ So the simulation techniques generate huge business profit. 8 See Generation_and_propagation_of_errors. 9 See Zheng-Liang Lu 150 / 179

18 All science is dominated by the idea of approximation. Bertrand Russell ( ) Essentially, all models are wrong, but some are useful. George E. P. Box ( ) Zheng-Liang Lu 151 / 179

19 Vectorization (Revisited) 10 ˆ The built-in functions such as sqrt(x) and exp(x) automatically operate on array arguments to produce an array result with the same size as the array argument x. 1 >> t = linspace(0, pi, 5); 2 >> y = sin(t) 3 4 y = More about vectorization. Zheng-Liang Lu 152 / 179

20 Advantages by Vectorization ˆ Appearance: vectorized mathematical code appears more like the mathematical expressions found in textbooks, making the code easier to understand. ˆ Less error prone: without loops, vectorized code is often shorter. ˆ Fewer lines of code mean fewer opportunities to introduce programming errors. ˆ Performance: vectorized code often runs much faster than the corresponding code containing loops. Zheng-Liang Lu 153 / 179

21 Performance Analysis In Real Time ˆ In addition to the theoretical analysis of algorithms, programmers can also use a timer to measure the performance. 11 ˆ Once you identify which functions are consuming the most time, you should determine why you are calling them, then look for alternatives to improve the overall performance. ˆ However, Amdahl s law 12 states that the speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program. 11 Note that the results may differ depending on the difference of run-time environments, so make sure that you benchmark the algorithms on the same conditions. 12 Amdahl (1967). Zheng-Liang Lu 154 / 179

22 Digress: Amdahl s Law ˆ Assume that a program has some codes which takes α% of total running time and can be further parallelized. ˆ Then the speedup can be achieved by investing p computers. ˆ More explicitly, Speedup = ˆ So we can calculate its limit, lim p α p α p (100 α) = + (100 α) 100 α. ˆ For example, the upper bound of speedup is 2 if α = 50. Zheng-Liang Lu 155 / 179

23 tic & toc ˆ The command tic makes a stopwatch timer start. ˆ The command toc returns the elapsed time ( 10 6 second) from the stopwatch timer started by tic. (Try.) 1 >> tic % Please wait for a second. 2 >> toc 3 4 Elapsed time is seconds. Zheng-Liang Lu 156 / 179

24 Tips for Performance 13 ˆ Preallocate arrays ˆ Repeatedly resizing arrays often requires Matlab to spend extra time looking for larger contiguous blocks of memory, and then moving the array into those blocks. ˆ Vectorize your code ˆ Create new variables if data type changes ˆ Use functions instead of scripts ˆ Avoid overloading Matlab built-in functions 13 See Techniques for Improving Performance. Zheng-Liang Lu 157 / 179

25 Example: A Benchmark 1 clear; clc; 2 3 order = 0 : 1 : 4; 4 t1 = zeros(1, length(order)); 5 t2 = zeros(1, length(order)); 6 t3 = zeros(1, length(order)); 7 8 for j = 1 : length(order) 9 num = 10 ˆ order(j); tic 12 y = []; 13 for i = 1 : num 14 y = [y, i ˆ 2]; % dynamic allocation of... array y 15 end 16 t1(j) = toc; Zheng-Liang Lu 158 / 179

26 17 18 clear y; 19 tic 20 y = zeros(1, num); % preallocation of array y 21 for i = 1 : num 22 y(i) = i ˆ 2; 23 end 24 t2(j) = toc; clear y; 27 t = 1 : 1 : num; 28 tic 29 y = t.ˆ 2; % vectorization 30 t3(j) = toc; 31 end Zheng-Liang Lu 159 / 179

27 t1 / t2 t1 / t3 t2 / t Speedup Array Size Zheng-Liang Lu 160 / 179

28 t1 / t2 t1 / t3 t2 / t3 Speedup (log scale) Array Size (log scale) Zheng-Liang Lu 161 / 179

29 All roads lead to Rome. Anonymous 但如你根本並無招式, 敵人如何來破你的招式? 風清揚 笑傲江湖 第十回 傳劍 Zheng-Liang Lu 162 / 179

30 1 >> Lecture 3 2 >> 3 >> -- Functions 4 >> Zheng-Liang Lu 163 / 179

31 Functions ˆ The first thing of algorithm design is to divide and conquer. ˆ In other words, a large and complicated problem would be conquered by solving its subproblems. ˆ In order to reuse the algorithms without copying the codes, the best way is to make them functions. 14 ˆ The idea of the (software) functions is similar to the math function, which is typically written in form of y = f (x) with the input x and the output y. 14 Make bricks for your castle. Zheng-Liang Lu 164 / 179

32 ˆ A function is a piece of computer code that accepts an input argument from the caller, and returns output argument for the specific job the caller dealing with. ˆ Functions allow you to modularize a program by separating its tasks into self-contained units. ˆ We can program more efficiently and avoid rewriting the computer code for calculations that are performed frequently Remember that the bug propagates when you copy and paste the codes. It is a serious problem especially when you are working on a medium- or huge-level project. Zheng-Liang Lu 165 / 179

33 Arithmetic Functions Zheng-Liang Lu 166 / 179

34 Trigonometric Functions 16 ˆ Recall that 1 rad = 180 π. 16 See Table in Palm, p Zheng-Liang Lu 167 / 179

35 Rounding Functions ˆ Can you organize an algorithm for the function ceil and floor? ˆ How to check if the input is an integer? (Try.) Zheng-Liang Lu 168 / 179

36 Discrete Math Functions Zheng-Liang Lu 169 / 179

37 Max Functions Zheng-Liang Lu 170 / 179

38 ˆ The function min works in a similar way to max. Zheng-Liang Lu 171 / 179

39 Functions for Central Tendency Zheng-Liang Lu 172 / 179

40 Variance and Standard Deviation Zheng-Liang Lu 173 / 179

41 Functions for Sizes Zheng-Liang Lu 174 / 179

42 Random Number Generators ˆ You may use randi(n, m, n) to produce an m-by-n random integer matrix ranging from 1 to N. Zheng-Liang Lu 175 / 179

43 ˆ You can generate a random number sampled from a standard uniform distribution, say by a linear congruential generator. 17 ˆ Be aware that there is no true random number generator in the machines. 18 ˆ Widely used in Monte Carlo simulation 19 and random number generation of other distributions 20. ˆ Use rng( shuffle ) to generate different random sequences. 17 See 18 For now, the modern computers are all deterministic. Quantum computers share theoretical similarities with non-deterministic and probabilistic computers. 19 See Glasserman (2003). 20 See the acceptance-rejection method and Metropolis-Hastings algorithm. Zheng-Liang Lu 176 / 179

44 User-Defined Functions ˆ A user-defined function is created by 1 function [outputvar] = funcname(inputvar) 2 % comment section 3 end ˆ The output variables, if there exist, are enclosed in square brackets. ˆ The input variables, if there exist, must be enclosed with parentheses. ˆ funcname should start with a letter, and be the same as the file name in which it is saved. ˆ Before this function can be used, it must be saved into the current folder If not, change the current folder or add to the path pool. Zheng-Liang Lu 177 / 179

45 Example: Addition of Two Numbers 1 function z = myadd(x, y) 2 % input: x, y (two numbers) 3 % output: z (sum of x and y) 4 z = x + y; 5 end ˆ It seems bloody trivial. ˆ Actually, the plus sign is a kind of syntactic sugar Recall how to use an addition instruction in assembly codes. Zheng-Liang Lu 178 / 179

46 Example: Mean of A Sequence 1 function y = mymean(x) 2 % input: x (array) 3 % output: y (mean) 4 5 sum = 0; 6 n = length(x); 7 for i = 1 : n 8 sum = myadd(sum, x(i)); % call myadd 9 end 10 y = sum / n; 11 end Zheng-Liang Lu 179 / 179